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(54) Video surveillance system 

(57) A video surveillance system for monitoring an 
area Is made up of a stationary pilot camera (12) for mon- 
itoring the area and one or more moveable slave cam- 
eras (16.18.20^) for monitoring at least part of the 
area. Each slave camera is allocated to part of the area 
being monitored. The pilot camera produces a signal 
representative of the area. The location of a moving 
object in the area monitored by the pilot camera is deter- 
mined. A signal is produced representing the location of 
tiie d^ect The slave cameras track the object based on 



the signal representing the location of the object The 
pilot camera is provided with a fisheye lens and thus has 
a field of view wider than that of the slave cameras. The 
area being monitored is divided into cartesian, polar or 
spherical coord nates thereby enabling a microcomp u ter 
to instruct the slave camera to follow the qbi9CL The sig- 
nal representative of the area is oompre&Md and trans- 
mitted over a comnunications channel for remote 
monitoring. 
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Description 
BACKGROUND 

The present invention relates generally to video sur- s 
veniance systems. Moreparticularty. it relates to a video 
surveillance system in which a pilot camera oontrdis the 
operation of one or more slave cameras. 

Known video sunreillance systems often monitor rel- 
atively large areas by means of several video cameras io 
or idome&' A dome may include one or more cameras* 
the operatian of wtiich can be remotely controDed. A 
camera in such a dome is typically equipped with a zoom 
lens which has a fimited field of view. The camera can 
also be panned and tilted by remote control. The extent is 
to which a particular camera or dome is able ID monitor 
an area is fimited by the field of view of its lens and the 
extent to which it can be panned and tilted. It is therefore 
possible for an intruder to avoid detection by a surveil- 
lance system if the can dodge the movement of the cam- 20 
era and keep out of its field of view. Thus, in order to 
minimize this problem, particularly when the monitored 
area is large, a system of the pricN- art uses a large 
nurft>er of domes operating under program control to 
cover the entire area. However, such systems do not ^ 
eliminate the risl( that an intruder will be undetected. In 
addition, such systems require a large number of cam- 
eras and a relatively complex control program in order to 
minimize the opportunity for intruders to escape detec- 
tion. 30 

SUMMARy OF THE INVENTION 

The present invention is a video surveillance system 
for monitoring an area The system is made up of a pilot 55 
camera for monitoring the area and a slave camera for 
nrwnitorihg at least part of the area. The pilot camera pro- 
duces a signal representative of the area. Means are pro- 
vided for determining the location of a nxyving object in 
the area monitored by the p9ot camera from the signal 4o 
representative of the area and for producing a signal rep- 
resenting the location of the otjecL Means are also pro- 
vided for causmg the slave camera to track tfie moving 
otiject based on the signal representing the k)catk3n of 
the object 4S 

In the preferred embodiment the pOot camera is sla- 
tk)nary and the slave camera is moveable. The pikit cam- 
era has afield of view wider than that of the slave camera 
To that end the pikit camera is prc^fcted wHh a fttheye 
lens. The signal containing the fisheye image produced so 
by the camera can t>e compressed and transmitted over 
a communicatkms channel to a remote monitoring sta- 
tkxi. The k)cation of the object isdetermlned tiy detecting 
tlie motion of the oK^ecL 

The invention also includes a method of monitoring ss 
an area corrprising the steps of: monitoring the area by 
means of a pilot camera, thereby producing a signal rep- 
resentative of the area; processing the vMeo output to 
determine the k)catk)n of a moving ot^ect in the area and 



producing a signal representative of the k>cation of the 
object; and controlling a slave camera such that the slave 
camera tracks the moving obj ect based on the signal rep- 
resentative of the kx:ation of the object. 

BRIEF DESCRIPTION OF IHB DRAWING^ 

Fig. 1 is a perspective view of a space nrK>nitored t>y 

the present InverrtkKi. showing a pik)t camera and 

slave cameras as well as the f ieU of view of the pilot 

camera and one slave camera: 

Fig. 2 is a diagram of the image produced by the pilot 

camera; 

Fig. 3 is a biock diagram of the control system of the 
present invenOort 

DETAILED DESCRIFnON 

The foik>wing is a descriptkxi of the preferred 
embodiment of the inventkxi. It is Interxjed to be illustra- 
tive of the invention rather than limiting. The full scope of 
the inventkm is to be detemnined by the appervled claims 
and their equvalents. 

In Fig. 1. the invention is shown installed in a Icuge 
room 1 0 such as a warehouse having ceiling 1 4, f k)or 24 
and walls 34. AstatkHiary pitot camera 12 is contained 
in pilot domell mounted on celling 14. Pitot camera 12 
isavkJeocameraofatypecomnxxilyused In vkJeo sur- 
veillance, except tfiat it has a lens 13 (see Fig. 3) liaving 
a very wkje angle of view, preferably a fisheye lens liav- 
ing a i\M of view of 180 degrees. The field of view of 
lens 13of pilot camera 12 is wMe enough for It to monitor 
an area 15 corresponding to the hatched area in Fig 1 
and covering the entire eoqmsB of f kx>r 24 and part of 
wails 34. 

Slave cameras 16. 18, 20 and 22 are mounted on 
ceiikig 14. Each of slave cameras 16, 18, 20 and 22 is a 
conventional vkleo aurveifianoe camera contained in a 
"doma" The term idome" rel^ here to an enclosure 
housing at least one camera capable of panning, tflting 
and zooming. The preferred dome is the SpeedDome 
available from Sensonnatic Bectronics Corp. of Deer- 
fieM Beach, Rorkia Each slave camera 16, 18, 20 and 
22 can be tilted, panned and zoomed by remote control. 
Cameras 16. 18, 20 and 22 respectively have lenses 17. 
19. 21 and 23 (See Fig. 3). Lenses 17, 19. 21 and 23 are 
preferably zoom lenses wtiich have a f ieU of view less 
than that of lens 13 of the pilot camera (ranging from 6 
to 60 degrees in a 10X lens). Each slave camera is allo- 
cated a portk)n of the total fieU of pik>t camera 12. Each 
camera 16, 18. 20 and 22 thus nnonitorB part of the total 
area 15 monitored pik>t camera 12. Cameras 16. 18. 
20 and 22-are respectively assigned to nx)nitor cells 26, 
28, 30 and 32 which are fbnned by divkfing f k)or 24 and 
walls 34 into sectors as sfiown in Figs. 1 and 2. While 
thefiekl of view of lenses 17. 19. 21 and 23 of cameras 
16. 18. 20 and 22 is f&r narrower than the full extent of 
each cell, the cameras can pan, tilt and zoom so each 
camera is able to cover the cell to which it is assigned 
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(i.e. a part of the total area monitored by the pilot cam- 
era). The ability to pan and tilt allows each slave camera 
to track the movement of an object (an intruder for eocanv 
pie) within the cell to which the camera is assigned an6 
into an adjacent cell. 5 

The invention is fllustrfited in more detail in Uockdia- 
gram form in Fig. 3. The system comprises a pilot dome 
1 1 containing a pilot camera 12 having fisheye lens 13 
and slave cameras 16. 18, 20 and 22, respectively hav- 
ing lenses 17, 19.21 and23.Suchcanoeiasareconven- io 
tionai PTZ camera devices available from Sensonnatic 
Electronics Corp. of Deerf ield Beach. Florida under the 
name SpeedDome. Such a camera incorporates a con- 
troller in the form of Z1 80 processor. Pfiot camera 1 2 arxl 
slave cameras 16. 18. 20 and 22 are connected to video u 
smtcher 40 which has video outputs 44. Switcher 40 is 
controlled by computer 41 via control line 43. Video out- 
put 44 of switcher 40 is fed into computer 41. which is 
provided with graphics adapter 42. Computer 41 can 
instruct switcher 40 to select any of cameras 12. 16, 18. 20 
20 and 22, the output of which can be displayed on mon- 
itor 54 with or without overtald graphics provided by 
graphics adapter 42. Video output 44 can also be con- 
nected to video cassette recorder 62 and monitor 63 as 
shown in Fig. 3. 25 

Computer 41 is programmed to instruct, via bus 45. 
any of camerasIS, 18. 20 and 22 to pan. tilt, zoom, focus, 
change f-stop etc. Slave cameras 16, 18. 20 and 22 are 
respectively equipped with controllers 25. 27. 29 and 31 
which cause the camera to pan. tilt and zoom under the 30 
control of computer 41. A user input 56 in the fomi 
orTouchtredQr" input device (available from Sensor- 
matic Electronics) or a mouse or tracMaall allows the user 
to control the operation of the slave cameras as well as 
switcher 40. An example of a system having the ability ss 
to control a plurality of cameras from an input dmce and 
to overlay graphics is the VIdeomanager System availa- 
ble from Sensormatic Electronics. 

Fig. 2 represents the image 60 produced by fisheye 
lens 13 of pilot camera 12. Pilot camera 12 is placed ver- 40 
ticalty above floor 24. Due to the fact that lens 1 3 is pref- 
erably a ffiheye lens which has an extremely wide angle 
of view (180 degrees), camera 12 is able to monitor all 
of cells 26. 28. 30 and 32. The pRot camera generates 
an electrical signal representative of the area it monitors 45 
and reproduces the scene within its field of view as a 
circular and flat fisheye Image 60 as shown in Fig. 2. In 
spite of the fact that waBs 34 are perpendicular to floor 
24, the f ish^ lens causes waDs 34 to appear flat but 
somewhat distorted at the outer edges of the image pro- so 
duced by camera 12. 

Any point in cells 26. 28. 30 and 32 can k>e descrik)ed 
by reference to a cartesian (or polar/spherical) coordi- 
nate system. Image 60 is made up of a targe number of 
pixels. Therefore, each pixel can be mapped to. and is ss 
identified by. a reference to the chosen coordinate sys- 
tem. A map of the coordinates versus the pixels in the 
image generated by camera 12 is stored in the memory 
of conputer 41 . Thus, for sitetantially every pixel corre- 



sponding to cells 26. 28. 30 and 32 within image 60. there 
is a unique coordinate stored in the memory of computer 
41. The location of an object in any of the cells can l;>e 
described reference to the coordinates corresporxiing 
to the location. Computer 41 is programmed to instruct 
controllers 25. 27. 29 and 31 respectively to direct slave 
cameras 16, 18, 20 and 22 to point at a particular point 
in the cell to which the particular slave camera is 
assigned. That point is identified by coordinates stored 
in the memory of computer 41 arxl corresponds to spe- 
cific pixels in image 60. Thus, for example, if an object 
70 (in this case an intrnder) at a location in cell 28 is 
detected by means of pBot camera 12. slave camera 18 
can be cSrected by computer 41 to pan and/or tilt arxi 
thus point at the location and to follow the movement of 
an o^ect 70 wfthm the cell to which It is assigned. Cam- 
era 18 tracks ol^ect 70 at least unta object 70 moves into 
an adjacent cell 30, whenooniputer41 instnids the cam- 
era assigned to that ceD (camera 20) to take over the 
surveillance from camera 18. In an alternative emtxxli- 
ment. more than one of slave cameras 16, 18. 20 and 22 
is instructed to track object 70 60 that H can be viewed 
from smral angles. 

In order to control a slave camera such that it tracks 
the nx)tion of an object such as an intruder, it is neces- 
sary to k)cate the object within the area monitored by the 
pilotcamera. Thisis preferably acoompTishedbya aimpie 
difference algorithm. Successive frames acquired by the 
pilot camem are stored cmd subtracted from one another 
t>y storffig the previous frame in RAM (frame store) 58 
and subtracting It from the current frame pixel pixel, 
Since the surroundings in which object 70 is found are 
jBut)stantially stationary, the sut>traction of successive 
frames will cancel out the surroundings, leaving only the 
moving object The image remaining after sutxtraction of 
subsequent frames is refen^ed to as a "delta image." This 
delta image, made up of a number of pixels, is continu- 
ously mapped to the coordinates stored 01 the memory 
of computer 41 which instructs the slave camera 
assigned to the cell In which the object is fbund 10 point 
at the object 

In order to account for relatively stow moving sur- 
roundings such as waving flags, plants etc.. the delta 
Imag e can t)e averaged over a number of frames or cal- 
culated from s^^eral previous frames (e.g. four previous 
frames). If the difference indicated by the delta image 
remains wHhin a certain maxurum movement or travel 
range, it can k>e ignored. If the (fiff erenoe indicated by the 
delta image is outside the maxinrum movement or range, 
a start signal will be generated so that computer 41 
instructs the appropriate slave camera to trackthe move- 
ment of the object. In additk)n. vUeo cassette recorder 
(VCR) 62 can be Instnicted to start recording on such 
detectkm of motion. Astorege device such asadiskdrive 
can be substituted for VCR 62. Thus, as any particular 
slave camera follows the movement of an object the 
image so acquired is stored by the VCR. As the ot>ject 
moves through successive cells, the output of the slave 
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camera assigned to the celt in which the object is found 
is fed into the VCR. 

An afternative way of detecting the location of ot^'ect 
70 is to use a pilot camera having a charge coupled 
device (CCD) sensitive to specific frequencies of radia- 
tion emitted t)y ot^ect 70, tor example, radiation in the 
infra-red spectrum. The slave cameras are then directed 
to the ooondinates wNch correspond to the pixels of the 
CCD which havedetected the specific infra-red radiation. 

Tbe use of a f isheye lens in the present invention 
makes the image captured by camera 12 highly suitable 
for compression and transmitted over a communications 
channel. While an object in the field of view of a fisheye 
lens moves, the background remains substantially still. 
This high temporal redundancy means that oonpressfon 
is more readily accomplished than wouU be the case 
with a oonventfonal FTZ camera since successive 
frames contain more irrformatfon in common vvith each 
other. In order to track a moving object with a conven- 
tional camera, the camera rruist be panned, tilted andAor 
zoomed. The image changes a great deal over time as 
the camera is panned, tated or zoomed since the back- 
ground appears to move as the camera moves. This 
makes compressfon more cunfoersome than in tiie case 
of the image generated by a camera having a fisheye 
lens. 

Compressfon hardware 64 Is connected direcUy to 
pilot dome 1 1 and to computer 41. This allows tiie raw 
video output of pifot camera 1 2 to be compressed. Com- 
pressfon hardware 64 is based upon a vkieo compres- 
sfon set available from Integrated Information 
Technotogy, Inc. of Santa Clara. Califomla under the 
name Visfon Controller/Vision Processor or equivalent 
circuitry. Compressfon schemes such as JPEG. MPEG 
or H. 261 can be implemented using this hardware The 
compressed vkleo signal produced by compression 
hardware 64 is fed into communfoations interface 66. 
Communications Interface 66 is a conventional interlace 
such asa modem or ISDN interfaca Both oompressfon 
hardware 64 and oonrvnunications Interfaoe 66 are con- 
trolled by computer 41. 

The above structure allows a large area to t>e mon- 
itored by tiie pilot camera and transmitted to a remote 
nx>nitoring station over a communications channel hav- 
ing a relatively limited barxfwidth. Conputer 41 instructs 
oompressfon hardware 64 to compress the vMeo signal 
generated by cantera 12 and instructs communications 
interface 66 to commence transmissfon of the com- 
pressed vfoeo image over oonmunlcations channel 68 
to remote nrxyutoring station 72. The image is deconv 
pressed and displayed at tiie monitoring station. The 
deconpressed imago can then t>e monitored for irrtrud- 
ers. While the decompressed fisheye image may not pro- 
vide resolution sufTictent to detect details such as tiie 
facial features of an intruder, when an intruder is 
observed appropriate action can be taken at the moni- 
taing station, for example, calling the poGce. In the 
meantime, computer 41 can instruct slave cameras 16. 



1 8. 20 and 22 to track tiie intmder and VCR 62 to record 
so that the intruder can be identified. 

Claims 

5 

1. A video surveillance system for monitoring an area, 
comprising: 

a pifot camera for monitoring ttie area and 
producing a signal representative of the area: 
10 a slave camera for monitoring at least part of 

the area; 

means for determining tiie focation of an 
object in tiie area monitored t>y the pilot camera from 
tiie signal representative of the area and for produc- 
ts ing a signal representing the focation of the ok^ect: 

nieansfbrcauang the slave camera to track 
tiie obi ect based on tiie signal representing tiie foca- 
tion of the object 

20 2. The system of daiml wherein tiie pifot camera has 
afieU of view wfoertiian tiie field of view of the slave 
camera. 

3. The system of daim 1 wherein tiie pilot camera is 
25 Stationary and ttie slave camera is movable. 

4. The system of daim 3 wherein the means for caus- 
ing ttie slave camera to track the object comprises 
means for panning or tilting the slave camera. 

30 

5. The system of daim 1 wherein tiie pifot camera has 
a fisheye lens. 

6. The system of daiml wherein the ok^ed is a moving 

ss otiject and the means for determintng the location of ^ 
ttie ob|ed comprises means for detecting the motion 
oftheok^ed. 

7. The system of daim 1 wherein ttie pifot camera pro- 
40 duces a plurality of images of ttie area and wherein 

ttie means for detecting the focation of the ok^ect 
comprise means for siftilracting successive images 
from each other. 

45 8. The system of daim 1 wherein the means for deter- 
mining the location of the obj act comprises means 
for detecting specfffo frequencies of radiation emtt- 
tedbytheofajecL 

60 9. Thesystemofdaiml further conprising means for 
compressing the signal representative of the area 
and means for transmitting the compressed signal 
over a commirtications channel. 

55 ia R>r use In a video surveillance system, a metiiod of 
monitoring an area conprising the steps of : 

monitoring the area by means of a pilot cam- . 
era thereby produdng a signal representative of ttie 
area; 



4 



7 



EP0 714 081 A1 



8 



processing the signal representative of the 
area to determine the location of an object in the 
area and producing a signal representative of the 
location of the object; 

controlling a slave camera such tiiat the slave 
camera tracks the object based on the signal repre- 
sentative of the location of the object 

11. The method of claim 10 wherein the pilot camera has 
a wider field of view than the slave camera. 

12. The method of daim 10 wherein the pilot camera is 
stationary and the slave camera is movable. 

13. Ihe method of daim 10 wherein the step of control- 
ling comprises panning a tilting the slave camera. 

14. The method of daim 10 wherein the Step of process- 
ing comprises a step of detecting motion in the 
object 

15. The method of daim 10 wherein the signal repre- 
sentative of the area comprises a plurality of succes- 
sive images and wherein the step of processing 
comprises a step of subtracting successive Images 
from each other. 

16. Themethodof daim 10 whereinthestepof process- 
ing comprises a step of detecting specific frequen- 
cies of radiation emitted by the ok^ect 

17. ThemethodofdaimlOfurtheroompristngthesteps 
of compressing the signal representative of the area 
and transmitling the compressed signal over a com- 
munications channel. 



21 . The system of claim 1 8 wherein the pilot camera pro- 
duces a plurality of images of the area and wherein 
the means fa detecting the location of the object 
comprise means for subtracting successive images 

5 from each other. 

22. The system of daim 18 wherein the means for deter- 
mining the location of the object comprises means 
for detecting specific frequences of radiation emit- 

10 ted by the object 

23. The system of daim 18 further comprising means 
for compressing the signal representative of the area 
and means for transmitting the compressed signal 

IS over a communications channel. 

24. A video surveillance system for monitoring an area, 
comprising: 

a pilot camera, comprising a f isheye lens, for 
20 produdng a signal representative Of the area; 

means for compressing the signal represent- 
ative of the area; 

means for transmitting the compr e ssed sig- 
nal to a remote monitoring station. 

25 

25. The system of daim 24 further comprising: 

a slave camera for monitoring at least part of 
the area; 

means for determining the location of an 
30 object in the area monitored by the pilot camera from 
the signal representative of the area and for produc- 
ing a signal representing the location of the object; 

means for causing the slave camera to track 
the ot^'ed based on the signal representing the loca- 
35 tionoftheot3ject 



18. A video surveillance system for monitoring an area, 
comprising: 

a pilot camera for monitoring the area and 
produdng a signal representative of the area, the 
pilot camera having a f isheye lens; 

a slave camera for monitoring at least part of 
the area: 

means for determing the location of an objed 
in the area monitored by the pilot camera from the 
signal representative of the area and for produdng 
a signal repreeenting the location of the object; 

meane for causing the slave camera to track 
the ot)ject based on the signal representing the loca- 
tion of the okajed by panning or tilting the slave cam- 
era. 

19. The system of daim 18 wherein the pilot camera is 
stationary and the slave camera is nmable. 

20. Thesystemofdaim18wherein1hemeansfordeter- 
mining the location of the d^ect comprises means 
for detecting motion of the ok)ject 



26. The system of daim 24 wherein the pilot camera is 
stationary and the slave camera is movabla 

40 Z7. The system of daim 26 wherein the means for caus- 
ing the slave camera to track the object comprises 
means for panning or tilting the dave camera. 

2a The system of daim 25 wherein the object is a mov- 
45 ing ot^ect and the means for determining the loca- 
tion of the object oonprises means for detecting the 
motion of the ot3ject 

29. Thesystemofdaim25whereinthepilotcamerBpro- 
60 duces a plurality of images of the area and wherein 
the means fa detecfing the location of the object 
comprise means fa subtracting successive images 
from each other. 

55 30. R)r use in an electronic surveillance system, a 
metlKxj of monitaing an area comprising the steps 
of: 

monitoring the area by means of a pilot cam- 
era comprising a f isheye lens, thereby produdng a 
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signal representative of the area; 

oonnpressing the signal representative of the 

area: 

transmitting the compressed signal to a 
remote monitoring station. s 

31. The method of daim 30 further comprising the steps 
of: 

processing the signal representative of the 
area to determine the location of an object in the io 
area and producing a signal representative of the 
location of the object: 

controlling a slave camera suc^ that the slave 
camera tracks the object based on the signal repre- 
sentative of the location of the object. 16 

32. The method of claim 31 wherein the pilot camera is 
stationary and the slave camera is mcvaUe. 

33. The method of claim 32 wherein the step of control- 20 
ring comprises panning or tilting the slave camera. 

34. The method of claim 31 wherein the object is a mov- 
ing object and the step of determining the location 

of the object comprises detecting the motion of the 25 
object 

35. The method of claim 31 wherein the signal repre- 
sentative of the area comprises a plurality of images 

of the area and wherein the step of detecting the 30 
location of the object comprises subtracting succes- 
sive images from each other. 

36. The system of daim 1 wherein the step of determin- 
ing the location of the object oonprises the step of 35 
detecting specific frequencies of radiation emitted 
l)ytheok4ect 
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